Comparing Bayes factors and hierarchical inference for testing general relativity with gravitational waves

In the context of testing general relativity with gravitational waves, constraints obtained multiple events are typically combined either through a hierarchical formalism or though multiplicative Bayes factor. We show that well-known dependence factors on analysis priors in regions parameter space without likelihood support can lead to strong confidence favor incorrect conclusions when one employs $\mathcal{O}(1)$ ambivalent as they depend sensitively priors, which rarely set principled way; additionally, $>\mathcal{O}({10}^{3})$ be conclusion depending many multiplied, and specifically much wider than underlying population. The instead infers ensemble distribution individual beyond-general-relativity does not suffer from this problem, generically converges correct conclusion. Rather naive multiplication, more reliable factor computed analysis. present number toy models showing practice multiplying conclusions.